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Abstract 

In this paper, we address the problem of controlling a system over an unreliable connection that 
is affected by time-varying delays and randomly occurring packet losses. A novel sequence-based 
approach is proposed that extends a given controller designed without consideration of the network- 
induced disturbances. Its key idea is to model the unknown future control inputs by random 
variables, the so-called virtual control inputs, which are characterized by discrete probability density 
functions. Subject to this probabilistic description, the actual sequence of future control inputs is 
determined and transmitted to the actuator. The high performance of the proposed approach is 
demonstrated by means of Monte Carlo simulation runs with an inverted pendulum on a cart and 
by a detailed comparison to standard NCS approaches. 



1. Introduction 

In networked control systems (NCS), communication between components of the control loop can 
be realized via a communication network instead of a transparent connection [H, 0). This system 
architecture offers many advantages, such as simple installation and maintenance, as well as a 
high flexibility in the system structure. Therefore, NCS can already be found in a wide range of 
applications, e.g., unmanned vehicles 0], telepresence systems 0], or mobile sensor networks 

However, it is well known that compared to a transparent connection, the presence of a commu- 
nication network in the control loop decreases the quality of control or even destabilizes the system 
0, Hi S] • This is mainly caused by time- varying transmission delays and randomly occurring packet 
losses, limited bandwidth of the communication channel, or quantization errors. Consequently, 
control methods for NCS have to consider both communication and control aspects. 

Our approach is based on the well-established control technique, named sequence-based control, 
which does not transmit just a single control input but a whole sequence of reasonable inputs for the 
future time steps. This idea takes advantage of the property of modern communication networks, 
in which data is transmitted in large time-stamped packets. The successfully transmitted sequences 
are stored in a buffer at the actuator and a specific selection logic enables that some reasonable 
inputs can be passed on to the plant at every time step. 

The main problem of this procedure is that the controller actually has to know the control 
inputs applied by the actuator in the past and in the future, in order to be able to determine the 
current optimal sequence of control inputs. However, this demand is obviously not realizable, due 
to unpredictability of the network-induced delays. 
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Figure 1: Considered NCS architecture. Controller and actuator are connected through a communication network, 
whereas the link between sensor and controller is transparent. For compensation of time delays and packet losses, a 
controller is employed, which transmits a whole sequence Uk of optimal control inputs instead of just a single one. 



1.1. Related Work 

In 3 and Q, a deterministic protocol is proposed to guarantee that the sequence of control 



inputs used by the controller for state predictions coincides with the sequence applied by the 
actuator. By enforcing this property, the so-called prediction consistency, there are some significant 
drawbacks. Especially in the case of long time delays, the controller is frequently in the so-called 
recovery mode, in which the actuator rejects inconsistently predicted sequences, even when they 
are based on recent measurements. 

In a scenario-based NCS controller is proposed calculating the optimal control inputs for 
each possible delay of the previously transmitted sequences. Then, the set of control sequences is 
transmitted to a smart actuator, which selects the correct sequence. Obviously, the complexity of 
this approach increases strongly when longer time delays occur. 

Many sequence-based NCS controller neglect communication aspects and send at every time 
step a sequence of inputs resul ting from an open-loop control problem exclusively depending on the 



current system state, e.g., see [12J] or [13J. Consequently, these approaches do not incorporate into 
the control decision the control inputs sent by the controller in the past and stored in the buffer 
of the actuator. But, these inputs also have potentially an effect on the future evolution of the 
system. 

A feedback controller designed for systems without network-induced disturbances is employed 
in 1J] for determining a control input sequence. Along with a deterministic model of the plant, 
future states and therefore, future control inputs are predicted, which are then transmitted in a 
control input sequence to the actuator. 

1.2. Key Idea 

In this paper, we extend any given state feedback controller designed without considering 
network- induced disturbances. The novel idea is to model the future inputs by random variables, 
named virtual control inputs. These random variables are characterized by discrete probability 
density functions over potential control inputs, which are derived from the data transmitted by the 
controller in the past. Based on this probabilistic description representing the best knowledge of 
the controller about the situation at the actuator, a sequence of control inputs for the future time 
steps is determined. 

1.3. Notation 

Throughout the paper, random variables a are written in bold face letters, whereas deterministic 
quantities a are in normal lettering. Furthermore, the notation a ~ /(a) means that the random 
variable a is characterized by its probability density function f{a). A vector- valued quantity a is 
indicated by underlining the corresponding identifier and matrices are always referred to with bold 
face capital letters, e.g., A. The notation a/% refers to the quantity a at time step k. Furthermore, 
auj denotes the quantity a at time step k based on information up to time t. The term mX r refers 
to a matrix with dimension m x r with all entries equal to and r is an abbreviation for rX r- 
Finally, the term r means rx i and I n denotes the identity matrix of dimension n x n. 
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1.4- Outline 

The remainder of the paper is organized as follows: In the next Section, the problem is defined 
and the assumptions made are listed. Then, the proposed controller scheme for NCSs is described 
in detail and stability properties are examined. Section VI presents simulation results with an 
inverted pendulum and compares the proposed approach to standard NCS techniques. A summary 
and an outlook on future work concludes the paper. 

2. Considered Problem 

Throughout the paper, we consider a discrete-time linear dynamic system described in state-space 
form via 

x k+1 = Ax k + Bu k + w k , (1) 

where x k £ W denotes the system state at time step k and u k £ R n the control input applied by the 
actuator. Note that due to time delays and packet losses in the network, u k is a random variable. 
The system noise is subsumed by w k ~ f w (w k ) and is assumed to be a zero-mean Gaussian noise 
process. Furthermore, the system matrices A € M sxs and B £ M sxn are assumed to be known. 

The components of the control loop are time-triggered, synchronized, and have identical cycle 
times. Furthermore, we assume that the actuator does not have sufficient calculation capacity to 
perform local control. 

In this paper, we restrict our considerations to the case, where the communication network is 
solely present in the controller-to-actuator link. We further assume that the controller has perfect 
information about the current system state x k of the plant, i.e., the system state is completely 
measurable by the sensor and the connection between sensor and controller is perfect. 

The employed network is capable of transmitting large time-stamped data packets and does 
not provide acknowledgements for successfully transmitted data as in so-called UDP-like protocols. 
The data transmission might be subject to time-varying and possibly unbounded^ delays, modeled 
as a discrete random process T k £ N. The realizations of this process describe how many time steps 
a packet generated in time step k will be delayed until it is received. It is assumed that r k ~ / t (t) 
is a white stationary process that is independent of w k and that the probability density function 
/ T (r) is known. 

Finally, we assume that a controller with a linear state feedback control law 

u k = ^-x k (2) 

is given that is designed without consideration of the network-induced disturbances. In the follow- 
ing, we propose a scheme that extends this given controller in such a way that it can deal with 
time delays and packet losses. 

3. Sequence-based Control 

In this section, we briefly review the general concept of sequence-based control as, e.g., used in 

since our control approach presented in the next section is based on this 
fundamental control concept. 

In sequence-based control, a controller generates not just a single control input for the current 
control cycle, but also control inputs for future N time steps (with JV £ N). The whole control 
input sequence is lumped into one data packet and sent over the network to the actuator. The 
actuator is equipped with a buffer, in which the most recent control input sequence is stored, i.e., 
that sequence that has the highest time stamp among all received packets. Therefore, when a new 



1 By allowing the time-delays to be unbounded, packet losses are incorporated into the description of the random 
delay processes since the loss of a packet corresponds to an infinite time-delay 
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packet is received by the actuator, it is taken into the buffer if its time stamp is higher than the 
one of the packet already stored in the buffer, otherwise it is neglected. Finally, in every time step, 
the actuator applies the appropriate control input of the buffered sequence to the plant, i.e., that 
control input of the sequence that corresponds to the current time step. 

For the following derivations, we need some further notations. A control input sequence gen- 
erated by the controller at time k will be denoted by An entry of that packet is denoted by 
Mk+m\k with m € {0, 1, N}, where the first part of the index (here: k + m) gives the time step, 
for which the control input is intended to be applied to the plant. The second part of the index 
(here: k) specifies the time step, when the control input was generated. For a packet of length 
N + 1 generated in time step k, this gives 

Uk = {«fc|fc,M&+i|&, • • • ,Uk+N\k} ■ (3) 

For example, let us assume the controller packet U k is received by the actuator at time step 
k + Tfc with Tfc € N. If none of the packets 

Uk+i, Uk+2, ■ ■ ■ , Uk+ Tk (4) 

has been received by the actuator so far, then the buffer is overwritten with the entries of Uk and 
the input u k + Tk \k is applied to the plant. Otherwise, if the actuator has received any packet from 
((3D until time step k + Tk, say, e.g., Uk+i, for i = l,...Tk, Uk is neglected and u k +T k \k+i °f tire 
buffered sequence Uk+i is applied. 

Since we do not assume that the time delays are bounded, it may happen that the buffer runs 
empty. In this case, the controller operates with a default input u d . 

It is obvious that the control inputs applied by the actuator depend on the packet delays as 
well as losses and, therefore, inherit the stochastic nature of the network. This gives rise to the 
stochastic control approach discussed in the next section. 

4. Sequence-based Control with Virtual Control Inputs 

It should be obvious from the preceding section that in sequence-based control, control inputs from 
packets sent in previous time steps may actively affect the future evolution of the plant. It therefore 
seems only reasonable to take these old control inputs explicitly into account, when calculating new 
control inputs, which is also the main idea of the proposed approach. 

In the following, we derive in Sec. 14. II a stochastic description of these already transmitted, but 
possibly applied control inputs, that we call virtual control inputs. Then, utilizing the concept of 
virtual control inputs, we design the controller in Sec. 14.21 

4-1. Virtual Control Inputs 

In this section, we introduce the novel concept of virtual control inputs. To that end, we first 
define the information set Ik that summarizes the information the controller can use at time step 
k to calculate Up.. Considering causal controllers, the information set includes all measurements 
and all control packets that were received and sent, respectively, until time step k. Furthermore, 
the information set contains the information about the given feedback matrix L, the dynamics of 
the system D given by Eq. (JTJ), the buffering logic B of the actuator described in Sec. and the 
stochastic characteristics of the process and measurement noise, so that 

lk = {x , ... ,x k ,U , ... ,Uk-i;-L,V,B,f w (m),f T (r)} . (5) 



Remark 4-1 The information that can be used by the controller, i.e., Ik, does not contain the 
knowledge that the controller will sent control input sequences in future. This means that the 
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Figure 2: Schematic illustration of the transmitted packets. Control inputs corresponding to the same time step are 
vertically aligned. For example, the potential control inputs that could be applied by the actuator at time step k are 
indicated by white rectangles. The last yellow entry is the default value that would be employed if the buffer runs 
empty. 



controller does not consider that in the next time step the state will be measured and a control 
sequence will be generated and sent to the actuator. This is justified by the fact that the proposed 
controller must not use this information for two reasons. First, the controller has to generate open- 
loop control sequences since the actuator has no access to state measurements. Second, if an entry 
of the control input sequence Uk+i is applied by the actuator, it is not possible that a control input 
of the sequence Uk will be applied afterwards. Therefore, the influence of future control sequences 
Uk+x (and following sequences) must not be considered in the generation of the control sequence 
U k . 

Based on X k , we define the virtual control inputs as follows. 

Definition 1^.1 (Virtual Control Inputs) A virtual control input u k+m \ k ~ f(u%. +m i k ) is a ran- 
dom variable that characterizes the control input Uk+m based on the information Zk (with k,m 6 Nj. 

Remark It is important to distinguish 1) Uk+m> that is a realization of the control input u k+m , 
2) u k+m \ k , that is an entry of the packet Uk and that describes the control input applicable at time 
step k + m calculated by the controller at time step k, 3) the virtual control input w^ +m | fc , that is 
a prediction of Uk+m based onZk, and 4) l^k+m\k ^ s rea ti za ti° n of Uk+m- 

To derive the probability density function f(u k+m \ k ) of the virtual control inputs u k+m , we note 
that, based on the information set Ik, there is only a finite set of discrete values of control inputs 
that could be applied by the actuator. This is illustrated in Fig. [5] for the case of N = 2, where 
the control inputs possibly applied at time step k are marked by white rectangles. It should be 
noted that, although this finite set of control inputs is discrete, the control inputs itself are over a 
continuous domain. The structure of the uncertainty can formally be described by a Dirac mixture 
density, so that it holds for the probability density functions of the virtual control inputs 

N 1 N-m 
f{ikk+m\k) = OL k+m\k ^(^fc+m|fc ~~ M ) + 2-s a k+m\k ^(^k+m\k ~ ^k+m\k-i) > 

i=0 

with 

N-m+l 

m€{0,l,..-,iV}, £ 4lm\k = ^ 

i=0 

where 5() is the Dirac delta function and o<- k+m i k are sca l ar weighting factors. 
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Remark 4-3 Since we do not make the assumption that the delays are bounded, it can occur that 
the actuator runs out of applicable control inputs. This is taken into account in f{u^. +m i k ) by the 

term u d . 

The weighting factors express the probability that the corresponding control input u k+m \ k _ i is 
applied by the actuator, i.e., 

a fe+m|fe = Floh (Uk+m = m+m^-ilW ■ 

The control input u.k+m\k-i ^ s applied by the actuator if the sequence buffered in the actuator at 
time step k + m has been generated by the controller k + m — {k — i) = m + i time steps ago. In other 
words, u k+m \ k _ i is applied by the actuator if the age of the buffered sequence, i.e., the difference 
between time step of generation and actual time step, at time step k + m is equal to m + i. In the 
following, we denote the age of the buffered sequence at time step k by 6 k . With this notation it 
holds that 



a 



(i) 



i\Xh 



^k+m\k = Pr °k(0 fc+ , 

Therefore, the weighting factors can be interpreted as estimates of 6 k . It is shown in the appendix 
that 9 k can be formulated as the state of a Markov chain (with state space {0, 1, 2, N + 1}) that 
is governed by the transition matrix P for that holds 



P(o, o) 

P(l, 0) 
P(2, 0) 



P(0, 1) 

P(l, 1) 
P(2, 1) 





P(l, 2) 
P(2, 2) 







P(2, 3) 



J>(N+1, 0) P(iV+l, 1) 

where the p( it j\ are equal to Prob '- 

'0 









P(N, N+l) 
P(N+1, N+l). 



P(N+1, 2) P(JV+1, 3) 

= j\9 k = i) and can be calculated by 



(6) 



P(i, i) = \ 1 ~ S Qr for j = i + 1 , 

r=0 



for j > i + 2 , 
for j = i 
for j < i 



Thereby, qi describes the probability of the event that a packet is delayed by % time steps. These 
probabilities can easily be derived since the probability density function of the time delays of the 



(i) 

network connection are known. Arranging the weighting factors CL k+m \ k in form of a vector 



a. 



k+m\k 



a 



(o) 

/c+m|fc' 



, Ot 



(N-m+1) 
k+m\k 



it holds that 



— k+m\k 





IiV+2-m 0(7v+2-m)xm 


(P m ) T -a fc | fc 


Tr (diag 


IjV+2-m 0(Af+2-m)xm 


(P m ) T -a k]k _ 


) 



(7) 



(8) 



where diag [s] denotes a matrix with the elements of the vector s on the diagonal and zeros every- 
where else, and the term Tr (S) denotes the trace of S. The term in the numerator represents a 
m-step future prediction of 9 k , where only the first N+2 — m elements are kept. The other elements 
describe the probability that future control inputs are applied, which are based on information that 
is not available, e.g., I k +i an d is therefore not used. The denominator normalizes the extracted 
subset of the predicted vector so that it sums up to one. 

According to Eq. (JS]), if cx k \ k is known, the other weighting factors a fc+1 | fc • • • a k+N \ k can be 
derived by means of the transition matrix P. To derive a k \ k we note that the state x k can also be 
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interpreted as the continuous- valued outputs of a Hidden Markov Model, that is governed by k . 
Hence, it is possible to apply the continuous-valued version of the Wonham filter [18| in the form 



f W {ilk ~ A^ifc-l ~~ ^lLk\k 
f w (x-k ~ - B« fc | fc _l 



H = diag 



(9) 



f w (x.h ~ Ax fe _! - Bu k]k _ N 
f w (x k - A^-i - Bu d 



(10) 



The prediction <x k \ k _i can be calculated using Eq. ([8]). It can be seen that a k \ k (and therefore 
SX.k+m\k) is time-varying. To reduce the complexity in the calculation, the weighting factors cx k \ k 
can be approximated by its stationary probability solution limk-s>oo£*fc|o = QLoo- The stationary 
solution can be computed by the equilibrium equation 



P T a 



that always has a unique solution according to Markov chain theory. Using instead of cx k \ k 
has the advantage that all weighting factors become time-invariant and the controller is easier to 
calculate. Furthermore, the stability of the closed-loop system can be analyzed more easily (see 
Sec. E]). We, therefore, derive the controller for both cases, but concentrate in the stability analysis 
on the time-invariant approximation. 

In section 14.21 we will need the expected value of the virtual control inputs. These can be 
calculated by 



^■fc+mlfc 



a 



— k+m\k f(^k+m\k) ^k+m\k 
(N-m+1) 



a 



(N-m+1) 
k+m\k 



k+m\k 
N-m 



±k+m\k 



k+m\k ^k+m\k-i 



N-m 

U d ) + a fc|m|fc *CS4*+ m |jfc -2kk+m\k-i) ) dul +m \k 



(12) 



i=0 



With the steady state approximation this becomes 

i=0 



N-m 



a TO Mfc+m|fc-i 



(13) 



4-2. Controller Design 

This subsection describes, how to design the sequence-based controller based on the linear 
feedback controller 

u k = 'L-x k , (14) 

where the feedback matrix L was designed for the plant ([!]) without consideration of network 
effects by, e.g., pole placement or another modern control method, such as LQR, H2, or Hoc. In the 
following, we use the feedback matrix L to generate control input sequences based on the predicted 
future states of the plant. 
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Based on the measured state x k at time step k, the entries of the control input sequence U k are 
calculated by 

Uk\k = L • x k , (15) 
Uk+i\k = L • E{x k+1 \ k } , (16) 

Uk+N\k = L • E{x k+N \ k } , (17) 

where x_k+m\k ~ f( x k+m\x k ) describes the state predictions conditioned on the information The 
state predictions are random with respect to the process noise and the virtual control inputs. The 
expected value of the predicted state predictions E{x k+m \ k } can be calculated by 

E{x k+1 \ k } = E{Ax k + Bu k + w k \l k } 

= Ax k + B • E{u k \l k } + E{w k \l k } 
= Ax k + B- E{ul k } 



( N 
Ax k + B • ajj +1) u d + a k\k mk-i 

\ i=0 



V 



Ax fc + B • a^ +1 ) u d + £ a« u k ^ , 



i=0 



= A • E{x k+1 \l k } + B • ^{Mfe+i + E{w k+1 \l k } 



(i) 

t k+l\k ^k+l\k-i 









iV-l 


* {N) u d 


+ E 


i=0 




N-l 


e } - d + 


E«: 




i=0 



-E'fec+mlfc} — E{Ax k+m _ l + Bu fc+m _ 1 + Mfc+m-1 \%k} 

= A • ^fe +m _ 1 | fe } + B • E{u v k+m _ 1]k } 

(N—m \ 
(N-m+1) d , (») \ 

a-l+m\k U + 2^ a k+m\k U k +m\k~i j 

/ N—m \ 

« A • Sfe +m _!| fc } + B • (a(T m+1 ) M d + E «S ^k+m\k-i) • (18) 

Remark 4-4 If the size of the packets is equal to 1, i.e., N = 1, then the proposed controller 
coincides with the given linear state feedback controller. 

For taking the expected value, we use (|12|) and the assumption that w_ k is zero-mean and indepen- 
dent of x k and y? k+m \ k . 

From (|15|) - (|17|) and (|18|) it follows that the extended controller using the time-invariant ap- 
proximation of the weighting factors is linear not only in the measured state x k but also in the 
control inputs of the sequences Uk, . . . , U k -N-i- Therefore, the controller can be formulated as 
linear feedback controller working on the augmented state 

U k = l-± k , (19) 
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where the augmented state is defined by 



with 



±k+l fc-2 



T T 



— A;+-?V— l|fe— lJ 
,.T IT 



T 

— fe|fc— iV 



T lT 
■fc+llfc— AT+lJ 



G 



n- 



iV(JV + l) 



(20) 



(21) 



The vector 77, contains all control inputs of the already sent control input sequences U^—i, Uj 



fe-2) 



. . ., Uk-N that still could be applied in time step k or later. The result (fT9"|) will be very useful in 
the next section, where a criterion for closed-loop stability for the extended controller is derived. 



5. Stability Issues 

In this section we will derive a criterion for closed-loop stability of the proposed controller. We 
concentrate on the case where the weighting factors of the virtual control inputs are approximated 
by its steady state distribution described by Eq. 

Remark 5.1 It should be noted, that, although an approximation is used in the controller design, 
the derived stability results for this controller are exact and not approximated. 

To derive the stability criterion, first, a model of network and actuator is derived, that, in a second 
step, will be combined with the model of the plant (TjQ) and the controller (|19|) . 

5.1. Combined Model of Network and Actuator 

Based on 77, and k as defined in section 14.14 the combined state space model of network and 
actuator can be formulated as 



Fry, + GU k , 



(22) 
(23) 
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where 5<o k ,i) is the Kronecker delta function defined as 



nxn(N-l) 
nxn(N-2) 











1 if 6 h = i 



9 ^ 1 if e k + 
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5.2. Model of the Closed-Loop System 

By using the augmented state xjs form ([2"0"j) and combining ([I]), (|2"2"j) . and 



, it holds 



.2-fc+l 



A 



B • Hfi 



B • J e 
G 



U, 



Using (fT9"j) results in 





A B H 6fc " 




B • Je fe • L 










G L 





m. k 



(24) 
(25) 

The closeddoop system described by (|25|) can be interpreted as an inhomogeneous Markovian jump 
linear system (MJLS). For this kind of system, several results on mean square stability are available 
in the literature , e.g ., and [ijj. In the following, we adopt the concept of mean square stability 
and proof from [191 ] . 

Definition 5.1 The system \25\) with Markovian jump parameter 9k is mean square stable (MSS), 
if for any initial condition 6q G {0, 1, • • • , N} and tjj^ € R d+S there exist a bounded \i € M. d+S and 
a symmetric positive- semidefinite matrix M (independent of and 6q) such that 



lim E\ip } = fi , 
lim EU -4> T X = M . 



(26) 
(27) 



Theorem 5.1 The system h25\) with Markovian jump parameter Ok and transition matrix P is 
stable in the mean square sense, if and only if 



r CT ((P 3 



diag[Aj ® Aj]) < 1 , 



(28) 



where r a (M) is the spectral radius of M and diag [Si] is the block diagonal matrix built by Sj in 
the diagonal with i = {0, 1, ...,N} and zero everywhere else, i.e., 



diag [S. 

••• S N 

Proof. The result follows from theorem 3.9 and 3.33 in [l 



S 
Si 








(29) 



□ 



6. Simulation Results 

In this section, we evaluate the presented method by means of simulations with an inverted pendu- 
lum on a cart, which is a classical benchmark for illustrating various control techniques. A basic 
description of this experimental setup can be found, e.g., in [211 ] . 



10 



t 

>. 



C 
Oh 



Mass of the cart 

Mass of the pendulum 

Friction of the cart 

Length to pendulum center of mass 

Inertia of the pendulum 



0.5 kg 
0.5 kg 
0.1 N/m/s 
0.3 m 
0.006 kg • m 2 



Table 1: Parameters of the inverted pendulum used in the simulations. 
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Figure 3: Probability density functions over the time delays of the two networks considered in the evaluation. 



6.1. Simulation Setup 

For realization of ([2]) , we use a classical LQR controller [22| • In Table [U the simulation param- 
eters of the inverted pendulum are shown. The weighting matrices are chosen with 



Q 



5000 




















100 














R = 


100 , 





and the continuous differential equation was sampled with a sampling time of 0.01 s. With this 
setting, the resulting state feedback matrix is 

L = [-6.54,-5.50,28.72,5.50] . 

At every time step k, we add a process noise to the position x_ k of the cart and to the angle 
4>k of the pendulum, which is characterized by a zero-mean Gaussian noise with varying standard 
deviation a w for different simulation runs. 

For all simulation runs, the initial state vector x is 



X ,X , (/),(/) 



[0,0.2, 0.2, 0] J 



In order to simulate the transmission characteristics of the network, two probabilistic models 
for the occurring time delays were employed, whose probability density functions can be seen in 
Fig. The time delays are bounded in both networks, because this allows a comparison to other 
sequence-based control methods, which need this assumption. 

Overall, we conducted 100 Monte Carlo simulation runs for each combination of selected stan- 
dard deviation of the process noise and selected probabilistic network model, where each run consists 
of 150 time steps. 

We compare the presented technique for NCS with virtual control inputs (VCI-NCS) to three 
other NCS control approaches. For better analyzing the quality of the compensation technique 
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0.02 




Figure 4: Example state trajectory for a single simulation run with network A and a standard deviation a w = 0.006 
of the process noise. The identifier x denotes the position of the cart and 4> the angle of the pendulum. The result 
of the controller without a network (CS) is depicted with a solid black line (j j) . the proposed approach VCI-NCS 
with a dotted blue (j j) . OL-NCS with a dashed red (j j) . and PC-NCS with a dashed dotted green line (j j) . 



for time delays, we consider a classical LQR (abbreviated by CS) with a transparent connection 
between controller and actuator. In this case, all calculated control inputs 



Uk\k 



■ %k\k 



are received by the actuator without any time delay. The control quality of CS can be seen as a 
ground truth for the NCS control methods. 

Furthermore, we compare VCI-NCS to a widely used NCS controller that sends at every time 
step a sequence of control inputs resulting from an open-loop control problem (OL-NCS) 
In more detail, the packet Uk sent in time step k contains the entries 



12, 1 



Hk\k 
Mfe+i|fe 



L ' %k\k ! 

L • x_k+i\k 



U.k+N-l\k 



L • 2Lk+N-x\k i 



where x k+i \ k is determined for 1 < i < N according to 

3Lk+i\k = \k + ^ILk+i— l\k ■ 

Finally, we implemented an instance of a class of NCS approaches that ensures the so-called 
prediction consistency by means of a deterministic protocol between actuator and controller. Since 
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/ 
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A 


98.57 


137.38 


140.02 


262.28 
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= 0.012 
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network 


A 


186.60 


275.22 


290.18 


619.88 
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= 0.001 


/ 


network 
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3.03 


3.42 


3.34 


5.37 


<Jw 


= 0.003 
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network 
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88.85 


129.68 


141.26 


317.16 


<Jw 


= 0.006 


/ 


network 
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110.37 


164.32 


166.49 


359.85 


a w 


= 0.009 


/ 


network 


B 


122.50 


158.54 


175.45 


386.63 


0~w 


= 0.012 


/ 


network 


B 


197.94 


275.24 


310.50 


588.27 



Table 2: Cumulated costs of the simulated state trajectories averaged over the 100 Monte Carlo runs for non-networked 
controller (CS), the proposed method considering virtual control inputs (VCI-NCS), the NCS method sending open 
loop control inputs (OL-NCS), and the approach based on prediction consistency (PC-NCS). 

we assume that successfully transmitted packets are not acknowledged by the network protocol, 
the parameter r max of the PC-NCS approach described in was set to the true maximal delay. 

6.2. Results 

In Fig. 21 an example state trajectory of a simulation run with network A and a standard 
deviation a w = 0.006 is depicted. The state trajectory of VCI-NCS is very similar to the one 
generated by the non-networked controller. Thus, the proposed method is an adequate technique for 
compensating time delays. In contrast, the trajectories of OL-NCS and PC-NCS are outperformed 
by VCI-NCS. 

In order to make quantitative statements, we conducted 100 Monte Carlo simulation runs with 
different parameter settings. The averaged costs over all runs are shown in Table [2J For small 
system noise, the cumulated averaged costs of the proposed approach using virtual inputs and the 
NCS method sending open loop sequences are very similar. This is based on the fact that in this 
case, the components of the Dirac Mixture densities characterizing the virtual control inputs do 
not differ strongly, or more precisely, the variances of these densities are very small. Thus, both 
methods use control inputs for generating the input sequences, which are very similar to the actually 
applied input. In contrast, if the system noise increases, the variance of the Dirac Mixture densities 
also increases. Considering the NCS approach sending open loop sequences, inputs used for the 
prediction of the system evolution do not coincide in general with the actually applied inputs. As 
a result, the quality of control decreases. By incorporating the potentially applied inputs in a 
stochastic way, the network-induced disturbances can be compensated better, even if the system 
noise is large. 

7. Conclusions 

We presented a novel sequence-based predictive control scheme for NCS that extends a given 
feedback controller to explicitly incorporate communication aspects as transmission delays and 
packet losses. The key idea of our approach is that the controller subsumes its knowledge about 
the control inputs potentially applied by the actuator in form of a discrete probability density 
function, the so-called virtual control inputs. Based on this probabilistic description, the controller 
determines sequences of future control inputs which are sent to the actuator. 

To the best of our knowledge, the concept of virtual control inputs is innovative and promising, 
especially since simulation results with an inverted pendulum show an excellent performance of the 
proposed approach in comparison to standard NCS methods. 

Future work will be concerned with incorporation of further information into the control decision. 
For example, promising aspects might be the investigation of 
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1. allowing for time- varying transmission characteristics of the network resulting in more ade- 
quate models of the real system, 

2. acknowledgements of successfully transmitted packets which allow to reduce the components 
of the virtual control inputs, 

3. closed- loop stability if the weighting factors of the virtual control inputs are estimated by the 
Wonham filter. 



Appendix: Derivation of the Transition Matrix 

The entries pu ^ (with i,j € {0, 1, • • • , N + 1) of the transition matrix can be categorized into 
three groups. The first group consists of the entries with j > i + 2, i. e., entries that are in the 
upper right triangle of P, which describe transitions from 9^ = i to Ok+i = j- These entries have 
to be zero as 9k, i-e. the age of the buffered control sequence, can only increase by one per time 
step due to the buffering scheme. 

The second group consists of the entries in the upper diagonal, i.e., the entries pu that 
describe the probability that the age of the buffered control sequence will increase by one. This 
corresponds to the case that the buffered sequence at time step k is not replaced and stays in the 
buffer. This will only occur if the actuator does not receive a packet that was generated after the 
actually buffered sequence. It therefore holds that 



P(i, i+1) 



IK 1 -*: 

j=0 



where qj denotes the probability that a packet that was generated j time steps ago and that has 
not been received yet, will be received during the next time step. It holds that 



Qj 



\r=j 



(30) 



where qi describes the probability of the event that a packet is delayed by i time steps, which can 
easily be calculated with the probability density function of the time delays given. Therefore, the 
first term in Eq, (|3(J|) describes the probability that the sequence generated i time steps ago will 
arrive during the next time step ignoring that we know that the sequence has not been received 
yet. The second term normalizes this probability by the probability that this sequence will arrive 
at all. Combining Eq.flHDD and Eq.flHDD it holds 



/ 



P(i, i+1) 



n(i-*) =n 

j=0 j=0 



'N+1 



1 - Qj 1 £ Qr 



\ 



n 

3=0 



1^ Qr 

r=j+l 
N+1 

\ r =j 



N+1 

Qr 

r=i+l 
N+1 

£ Qr 

r=0 



N+1 i 

H 1 r = 1 - 1 r 

r=i+l r=0 



(31) 



The remaining entries pu j\ with j < i build the lower triangle of P and describe transitions 
where 9k does not increase. This corresponds to the case when the buffered sequence is replaced 
by a newer one. The probability that this happens is given by the probability that a new packet is 
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received (that was generated after the buffered sequence) and all packets generated after this new 
packet are not received. This means 

3-1 

Hi, 3) = QjYli 1 -^) (32) 

r=0 

Note that in the above equation the probabilities whether a packet is received in the next time step 
is conditioned on the information that the corresponding packet has not been received yet. This 
condition is necessary since, if neglected, the transition could not start in state Ok = i- Using Eq. 
([50]) and Eq. it holds that 

3-1 (N+l \ _1 iV+l 

pa, 3) = & ~ &o = Qj ( \ qr qr = n ■ ( 33 ) 

r=0 \ r=j ) r=j 
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